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FOREWORD 


This  research  and  development  effort  was  conducted  under  advanced  development  program 
element  0603720N  (Education  and  Training),  project  work  unit  R1772  (Education  and  Training 
Development),  Task  ET003  (Skill  Enhancement  Program)  and  sponsored  by  the  Chief  of  Naval 
Operations  (OP-01).  The  objective  of  this  work  unit  is  to  identify  and  develop  instructional 
technologies  and  technologies  that  effectively  correct  identified  skill/knowledge  deficiencies  for 
broader  use  in  Navy  technical  training. 

The  objective  of  this  effort  was  to  identify  factors  related  to  student  studying  in  Navy  enlisted 
schools,  and  then  design  and  evaluate  a  functional  research  instrument  to  assess  those  factors.  This 
report  was  originally  published  in  Educational  Technology  Research  &  Development*  and  is  being 
reprinted  by  the  Navy  Personnel  Research  and  Development  Center  (NAVPERSRANDCEN)  for 
wider  military  distribution. 


THOMAS  F.  FINLEY  RICHARD  C.  SORENSON 

Captain,  U.S.  Navy  Technical  Director  (Acting) 

Commanding  Officer 


'Seymour,  G.  E„  Main,  R.  E„  Randel,  J.  E.,  &  Morris,  B.  A.  (1991).  Study  factors  and  their  impact  on  military 
school  performance  measures.  Educational  Technology  Research  A  Development,  39  (2),  19-28. 
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SUMMARY 


Background 

Many  Navy  enlisted  “A"  schools  have  traditionally  encountered  small  groups  of  students  in 
each  class  who  have  experienced  academic  difficulties.  That  is  not  surprising  given  that  similar 
problems  are  being  reported  by  colleges  and  employers.  In  order  to  understand  the  dynamics  of 
academic  performance,  research  was  initiated  into  student  attitudes,  behaviors  and  other  study 
factors  that  may  influence  their  academic  achievement 

Methodology 

During  an  iterative  process,  Navy  Personnel  Research  and  Development  Center  researchers 
devised  a  model  of  study  activity  and  related  factors  based  on  military  research  reports  and  their 
own  experiences  in  military  school  settings.  The  goal  was  to  identify  and  define  all  possible 
categories  of  student  behaviors  or  factors  that  relate  to  study  performance.  Once  the  initial  set  of 
categories  was  developed,  independent  lists  of  possible  questions  were  written  and  categorized. 
Group  meetings  served  to  simplify  and  clarify  both  the  dimensions  and  questions  for  use  in  a  self- 
report  questionnaire.  The  Survey  was  administered  at  five  Navy  “A”  schools,  and  evaluated  at  two 
of  them. 

Findings 

The  reliability  of  the  Survey  instrument  was  acceptable  but  marginal.  When  the  two  validity 
studies  are  viewed  together,  a  pattern  emerges.  Of  the  16  Study  Factors,  1 1  were  significant  in  one 
or  both  schools.  What  is  most  conspicuous  in  this  pattern  are  the  findings  for  memorization, 
concentration,  and  questioning.  Memorization  was  related  to  test  performance  in  both  schools  and 
was  related  to  academic  probation.  Likewise,  both  concentration  and  questioning  were  related  to 
test  performance  in  both  schools. 

Recommendation 

These  Study  Factors  deserve  attention  in  Navy  technical  training  schools.  Disregarding  costly 
academic  attrition  and  setbacks,  even  students  who  (barely)  pass  the  tests  may  be  marginal 
performers  once  they  arrive  in  the  fleet.  To  make  enlisted  schools  more  effective.  OP-01  should 
sponsor  an  effort  to:  (1)  improve  the  reliability  of  the  Survey,  (2)  assess  the  generality  of  the 
findings,  and  (3)  investigate  whether  study  factor  training  can  be  implemented  to  influence  test 
performance. 
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A  16-dimension  survey  to  assess  study  skills 
ms  designed  and  administered  to  705 
students  enrolled  in  five  Navy  schools.  After 
revision  based  on  reliability  analyses,  valid¬ 
ity  analyses  were  performed  at  two  of  the 
schools  by  correlating  survey  responses  with 
test  performance  measures.  At  one  school, 
correlations  indicated  that  those  students 
who  were  more  competitive  or  motivated, 
had  higher  concentration  or  memorization 
skills,  or  asked  more  questions  performed 
significantly  better  on  the  most  difficult  tests 
in  the  school.  At  the  second  school,  using 
partial  correlations  to  control  for  student 
ability,  study  factors  reliably  predicted  test 
performance,  irrespective  of  student  ability. 
Multiple  regression  coefficients  of  .618  and 
.379  supported  the  independent  contribution 
of  several  study  factors  to  test  performance. 
Selected  study  skill  training  resources  used 
by  the  military  are  identified  and  functional 
research  directions  are  described. 
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□  In  traditional  academic  settings,  personal 
aids  or  skills  to  enhance  learning  usually  have 
been  limited  to  reading  comprehension,  mem¬ 
ory,  or  prose-learning  content  areas.  The  pri¬ 
mary  reason  is  that  both  teachers  and  theorists 
have  long  recognized  that  reading  and  its  com¬ 
prehension  are  fundamental  to  the  learning 
process  (Wittrock,  1974).  Thus,  prior  to  1940 
the  primary  topics  for  learning  how  to  study 
consisted  of  time  mangement,  systematic  note 
taking,  outlining  or  underlining,  and  selecting 
a  place  to  study  (Schulte  &  Weinstein,  1981). 

A  major  departure  from  this  limited  focus  of 
study  skills  occurred  with  the  formal  presenta¬ 
tion  and  then  later  acceptance  of  Robinson's 
(1946)  Survey-Question-Read-Rerite-Review 
(SQ3R)  method.  (Robinson  referred  to  it  as  the 
"Survey  Q3R  Method.”)  Although  the  SQ3R 
method  proved  extremely  useful  for  address¬ 
ing  specific  deficits  in  studying,  education  the¬ 
orists  recognized  that  the  composite  of  study 
skills  was  much  more  extensive  and  complex 
than  previously  thought.  In  particular,  the  in¬ 
vestigation  of  cognitive  issues,  as  opposed  to 
simple  individual  differences  and  behavioristic 
approaches,  became  more  pronounced. 

Starting  in  the  mid-1970s,  the  U.5.  military 
services  funded  several  learning-skills  research 
projects  that  were  to  occupy  a  prominent 
place  in  the  scientific  literature.  For  example, 
Dansereau,  Actkinson,  Long,  and  McDonald 
(1974),  under  contract  with  the  Air  Force, 
reviewed  the  research  literature  related  to  aca¬ 
demic  learning.  From  that  basis,  Dansereau 
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and  his  colleagues  (e  g.,  Dansereau  et  al., 
1979)  developed  their  Learning  Strategy  Sys¬ 
tem,  which  they  partitioned  into  two  parts — 
Primary  Strategies  and  Support  Strategies. 
The  Primary  Strategies  consist  of  Compre¬ 
hension-Retention  and  Retrieval-Utilization, 
each  of  which  contains  the  following  compo¬ 
nents:  understanding,  recall,  digesting/detail¬ 
ing,  expansion,  and  review.  Adding  the  letter 
"M"  from  "mood"  to  the  first  letter  of  these 
Primary  Strategies  provides  the  well-known 
acronym,  MURDER.  The  Support  Strategies 
consist  of  goal  setting,  concentration  manage¬ 
ment,  monitoring  and  diagnosing,  mood  set¬ 
ting,  and  mood  maintenance. 

Shortly  thereafter,  in  the  late  1970s  and  early 
1980s,  Weinstein  and  her  co-workers  (Wein¬ 
stein  et  al.,  1980;  Weinstein,  Washington, 
Wicker,  Duty,  St  Underwood,  1980)  developed 
a  set  of  cognitive  learning  strategies  for  the 
Army.  Like  Dansereau,  Weinstein  developed 
a  comprehensive  model  of  the  teaching¬ 
learning  process,  only  one  part  of  which 
involves  study  skills.  From  a  study  skills  pen 
spective,  Weinstein  and  Mayer  (1986)  defined 
eight  learning  strategies  identified  as  basic 
rehearsal,  complex  rehearsal,  basic  elabora¬ 
tion,  complex  elaboration,  basic  organization¬ 
al,  complex  organizational,  comprehension 
monitoring,  and  affective  and  motivational 
strategies.  Currently  the  Army  has  incorpo¬ 
rated  five  learning  strategies  into  its  Job  Skills 
Education  Program:  time  management,  read¬ 
ing  strategies,  problem  solving  (mathematics), 
test  taking,  and  motivational  skills  (Hoffman, 
Hahn,  Hoffman,  it  Dean,  1988;  Wilson,  1990). 
The  work  of  Brown  and  her  co-workers  (e  g., 
Brown,  Bransfbrd,  Ferrara,  te  Campione,  1983) 
on  metacognition  also  has  stimulated  both 
research  and  application  in  this  area. 

The  Navy  took  a  different  tactic.  In  a  series 
of  studies  in  Navy  schools,  researchers  dif¬ 
ferentiated  two  types  of  reading,  called  Read¬ 
ing  to  Do  and  Reading  to  Learn.  Students 
were  asked  to  report  which  activities  "had 
helped  them  to  learn  the  information  in  a 
reading-to-learn  task"  (Sticht,  1979,  p.  279). 
Four  major  categories  were  identified:  reread- 
rehearse,  problem  solve-question,  relate- 
associate,  and  focus  attention.  In  his  review 
of  cognitive  psychology  research  in  the  mili¬ 
tary,  Wittrock  (1979,  p.  309)  commented  that 


"the  armed  services  are  in  the  forefront  of 
application  of  principles  of  cognitive  psychol¬ 
ogy  to  instruction." 

Although  the  military  services  have  been 
productive  in  this  area,  there  has  been  no  defi¬ 
ciency  of  academic  research  activity  on  the 
topic  of  study  skills.  This  research  has  resulted 
in  the  development  of  several  instruments  to 
assess  study  skills.  Among  these  are  the  Moti¬ 
vated  Strategies  for  Learning  Questionnaire 
(MSLQ)  (Pintrich  et  al.,  1988),  the  Study  Atti¬ 
tudes  and  Methods  Survey  (SAMS)  (Bache¬ 
lor  k  Michael,  1988;  Nadson,  Michael,  St 
Michael,  1988);  the  Survey  of  Study  Habits  and 
Attitudes  (SSHA)  (Davou  St  McKelvie,  1984; 
Gadzella,  1982);  the  Study  Skills  Question¬ 
naire  (SSQ)  (Bartlett  it  Knoblock,  1988);  and 
the  Study  Process  Questionnaire  (SPQ)  (O'Neil 
k  Child,  1984).  Derry  and  Murphy's  (1986) 
review  of  learning  methods  training  confirmed 
that  the  study  skills  topic  and  research  are 
expanding. 

Nevertheless,  several  major  considerations 
confront  the  topic  of  study  skills  assessment 
and  evaluation.  One  of  these  is  the  absence 
of  a  consensus  concerning  the  domain  of 
study  skills.  In  addition,  many  of  the  current 
diagnostic  instruments  address  issues  that  are 
not  germane  in  all  settings.  For  example, 
many  such  instruments  assume  that  a  student 
is  taking  general  education  courses  instead  of 
one  course,  as  is  common  in  military  or  tech¬ 
nical  schools.  Others  assume  more  flexibility 
in  study  time  than  most  military  students 
have.  Finally,  some  study  skills  surveys— like 
the  SSQ,  because  of  its  specialized  vocabulary— 
were  designed  primarily  for  administration  by 
specialists.  For  these  reasons,  a  new  instru¬ 
ment  to  assess  student  skills  related  to  the 
study  process  was  devised  and  evaluated  in 
military  contexts.  Further,  Wade  and  Trathen 
(1989)  reported  that  most  of  the  early  research 
on  study  methods  failed  to  assess  the  relation¬ 
ship  between  study  techniques  and  recall. 
Lastly,  Weinstein  and  llnderwood  (1985) 
reported  that  many  research  studies  have  used 
grade-point  average  (GPA)  as  a  criterion. 
Whereas  GPA  is  an  unambiguous  cnterion, 
traditionally  it  embodies  several  deficits, 
namely,  course  self-selection,  a  wide  variety 
of  course  requirements,  and  diverse  grading 
systems. 
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These  issues  were  addressed  in  the  research 
described  in  this  article.  More  generally,  this 
report  describes  the  development  of  a  study 
skills  survey  and  four  related  research  studies— 
two  addressing  its  reliability  and  two  validity 
studies. 


INSTRUMENT  DEVELOPMENT 

During  an  iterative  process,  NPRDC  research¬ 
ers  devised  a  model  of  study  activity  and 
related  factors  based  on  military  research 
reports  and  their  own  experiences  in  military 
school  settings.  The  objective  was  to  identify 
and  define  all  possible  categories  of  student 
behaviors  or  factors  that  relate  to  study  per¬ 
formance.  Once  the  initial  set  of  categories 
was  developed,  independent  lists  of  possible 
questions  were  written  and  categorized. 
Group  meetings  served  to  simplify  and  clarify 
both  the  dimensions  and  questions  for  use  in 
a  self-report  questionnaire.  During  the  pro¬ 
cess,  the  number  of  dimensions  was  ex¬ 
panded  and  the  questions  were  modified. 
Notable  goals  during  this  process  were:  (a) 
item  development  and  revision  consistent 
with  the  study  factor  definitions,  to  address 
the  continuously  revised  model,  (b)  preserv¬ 
ing  conceptual  distinctions  between  the  di¬ 
mensions,  (c)  maintaining  a  low  requirement 
for  readability,  so  that  the  eventual  survey 
could  be  used  widely,  (d)  keeping  questions 
as  short  as  possible,  in  consideration  of  the 
respondent,  and  (e)  use  of  a  single  ana  sim¬ 
ple  response  format. 

Study  factors,  not  just  study  skills,  served 
as  the  focus  because  important  aspects  of  an 
individual's  study  behavior  may  not  be  clas¬ 
sified  as  a  skill.  A  good  example  is  test  anxi¬ 
ety.  Spielberger  (e  g.,  Spielberger,  Gonzalez, 
&  Fletcher,  1979)  has  been  particularly  active 
in  promoting  the  concept  of  test  anxiety  as 
an  essential  component  of  learning  strategies. 
Additionally,  although  some  researchers  may 
not  consider  motivation  a  study  skill,  both 
Dansereau  (1985)  and  McCombs  and  Dob- 
rovolnv  (1982)  have  effectively  used  motiva¬ 
tional  training  in  their  systems.  Similarly, 
McKeachie  and  his  co-workers  (e.g.,  Pintrich 
et  al. ,  1988)  emphasize  a  motivational  com¬ 
ponent  because  their  Motivated  Strategies  for 


Learning  Questionnaire  has  55  motivational 
items  and  55  cognitive  items.  Given  that  fac¬ 
tors  other  than  study  skills  have  been  impor¬ 
tant  for  study,  this  instrument  was  identified 
as  the  NPRDC  (Navy  Personnel  Research  and 
Development  Center)  Study  Factors  Survey. 

Table  1  lists  the  16  conceptually  indepen¬ 
dent  study  factors  and  their  operational  defi¬ 
nitions.  Whereas  all  16  study  factors  are 
conceptually  distinct,  some  dimensions  were 
expected  to  be  related  empirically,  perhaps  in 
different  populations,  tor  two  reasons.  First, 
one  dimension  often  may  imply  another 
dimension.  For  example,  memorization  may 
be  much  easier  and  thus  more  useful  for  stu¬ 
dents  who  do  not  have  test  anxiety.  Second 
and  more  importantly,  students  with  good 
academic  backgrounds  may  develop  and  use 
several  of  these  study  factors  automatically, 
whereas  students  with  poor  academic  back¬ 
grounds  might  use  few  or  none  of  them. 
Although  the  development  of  distinct  factors 
was  a  goal,  the  conceptual  differentiation  of 
characteristics  among  the  study  factors  was 
viewed  as  the  more  critical  objective. 

The  unadministered  version  of  the  NPRDC 
Study  Factors  Survey  consisted  of  16  study  fac¬ 
tors  as  assessed  by  98  items  using  a  4-point 
response  scale  that  ranged  from  Almost  Always 
(4)  to  Almost  Never  (1).  This  number  of  factors, 
16,  is  larger  than  typically  reported  by  other 
study  skills  researchers;  however,  the  level  of 
specificity  in  each  researcher's  model  usually 
accounts  for  such  differences.  The  survey  was 
evaluated  for  both  reliability  and  validity.  Each 
of  the  two  reliability  estimations  and  two 
methods  of  validity  assessment  are  described 
individually  below. 


INITIAL  RELIABILITY  ASSESSMENT 

Method 

Subjects 

Because  the  scales  had  no  previous  empirical 
analysis,  reliability  was  assessed  using  a  two- 
step  process.  The  survey  was  initially  admin¬ 
istered  at  two  Navy  schools  (N  =  117  and  127) 
that  train  new  enlistees  in  electrical/electronic 
occupations  following  their  entry-level  train- 
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TABLE  1  □  Study  Factor  Definitions 


Anxiety 

Competition 


Concentration 

Elaboration 

Graphic  Study  Aids 
Group  Study 
Mastery  Beliefs 
Memorization 

Motivation 

Organization 

Questioning 

Review 

Self-Monitoring 

Study  Resource  Management 
Test  Anticipation 
Test  Strategi 


Generalized  fear  associated  with  learning  or  testing. 

A  tendency  to  compare  one's  performance  to  that  of  classmates.  The 
perception  of  one's  performance  as  being  evaluated  m  comparison 
to  others. 

The  ability  to  focus  on  learning  despite  either  internal  or  external 
distractions. 

The  use  of  any  of  several  methods  to  integrate  and  understand 
information. 

The  appreciation  and  use  of  charts,  figures,  and  tables  for  study. 

The  tendency  to  interact  with  other  students  during  study. 

Attitudes  relating  academic  achievement  to  personal  effort. 

The  use  of  any  of  several  mental  devices  to  store  and  later  recall 
information. 

The  level  of  drive  or  perceived  incentive  to  either:  (a)  complete  training 
at  this  school  or  (b)  learn  assigned  training  materials. 

The  use  of  structure  and  preparation  of  materials  to  be  learned. 

The  verbal  process  of  seeking  clarification. 

A  selective  examination  of  previously  read  course  materials  to  enhance 
learning. 

A  mental  review  of  cognitive  process  to  ensure  it  is  logical  and 
compatible  with  previous  knowledge  and  assumptions. 

The  foresight  to  maximize  environmental  resources  to  permit  study. 

The  ability  to  predetermine  test  content. 

The  use  of  techniques  during  a  test  to  maximize  the  number  of  correct 
responses. 


TABLE  2  □  Two  Reliability  Estimates  of  the  NPRDC  Study  Factore  Survey 


Scale 

TWO  SCHOOLS 

ALL  FIVE  SCHOOLS 

Items 

Alpha 

Alpha 

Mean 

S.D. 

Anxietv 

5 

.662 

.712 

11.66 

2.94 

Competition 

3 

.657 

.626 

8.74 

2.23 

Concentration 

7 

.649 

.669 

19.51 

3.30 

Elaboration 

6 

.734 

.731 

15.40 

3.41 

Graphic  Study  Aids 

3 

.681 

.680 

9.10 

1.79 

Group  Study 

5 

.910 

.903 

11.06 

4.01 

Mastery  Beliefs 

7 

.609 

.585 

16.66 

2.35 

Memonzation 

5 

.600 

.632 

1465 

2.60 

Motivation 

8 

.738 

.732 

27.10 

3.41 

Organization 

8 

.748 

.728 

22.39 

4.14 

Questioning 

3 

.729 

.727 

9.86 

1.94 

Review 

5 

.718 

.650 

15.88 

2.50 

Self-Monitonng 

8 

.706 

.668 

22.69 

3.51 

Study  Resource  Management 

5 

.727 

.716 

14.68 

2.81 

Test  Anticipation 

5 

.676 

.645 

14.07 

2.67 

Test  Strategy 

3 

.594 

.625 

8.95 

2  16 

N  range 

237-241 

688-700 
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ing  in  boot  camp.  The  typical  student  in  these 
two  schools  is  just  out  of  high  school,  has  a 
high  school  diploma,  and  may  be  considered 
an  average  or  slightly  below  average  student. 

Results 

Based  on  the  initial  administration  of  the 
Study  Factors  Survey,  items  were  deleted  from 
the  scales  until  no  further  improvement  in  reli¬ 
ability  estimates  was  obtained.  No  item  was 
added  to  any  scale.  After  item  deletion,  the 
values  of  coefficient  alpha  for  the  16  scales 
ranged  from  .594  to  .910  and  the  number  of 
items  per  scale  ranged  from  3  to  8.  These  sta¬ 
tistics  are  shown  in  the  second  and  third  col¬ 
umns  of  Table  2. 


SECOND  RELIABILITY  ASSESSMENT 
Method 
Subjects 

As  part  of  a  larger  Navy-wide  project,  NPRDC 
Study  Factor  Survey  forms  were  administered 
at  and  received  from  three  additional  Navy 
schools  (,V  =  62,  98,  and  301,  reflecting  their 
relative  enrollment  rates)  These  schools  also 
train  enlisted  personnel  in  a  variety  of  tech¬ 
nical  occupations  immediately  following  boot 
camp  training 

Results  and  Discussion 

Table  2  also  shows  the  coefficient  alpha  esti¬ 
mates  of  internal  consistency  that  were  ob¬ 
tained  using  the  data  from  all  five  Navy 
schools  and  the  scale  means  and  standard 
deviations  obtained  for  these  schools.  As 
shown,  the  coefficients  fluctuated  somewhat 
between  the  first  and  second  reliability  assess¬ 
ments;  however,  most  remained  relatively 
stable.  For  the  entire  sample,  coefficient  alpha 
ranged  from  .585  to  903,  whereas  most  values 
were  in  the  moderate  .6  and  .7  ranges  The 
form's  total  internal  consistent  estimate  was 


.918.  Adding  items  that  had  been  deleted  in 
the  previous  reliability  analysis  did  not  in¬ 
crease  the  more  recent  alpha  estimates. 

Whereas  a  few  of  the  scales  demonstrated 
relatively  low  reliability  estimates,  most  of 
them  are  considered  acceptable  or  very  good. 
The  a  priori  dimensions  are  logically  compat¬ 
ible  with  the  results  from  the  internal  consis¬ 
tency  analysis.  Moreover,  these  reliability 
estimates  are  consistent  both  with  the  previ¬ 
ous  analysis  and  with  the  reliability  estimates 
for  commercial  study  skill  instruments  as 
reported  in  Weinstein  and  Underwood  (1985). 

FIRST  VALIDITY  STUDY 

Method 

Generally,  students  in  Navy  enlisted  schools 
attend  classes  daily  and  are  tested  at  least  once 
a  week.  There  are  typically  from  20  to  25 
multiple-choice  items  per  test.  The  tests  are 
knowledge  based  and  performance  is  mea¬ 
sured  using  percentage  correct.  Test  perfor¬ 
mance  standards  are  fairly  rigid,  typically  80% 
correct.  Those  students  who  do  not  meet  test 
performance  standards  usually  are  either  put 
on  academic  probation,  set  back  to  another 
class,  or  dismissed  from  the  school.  For  this 
study,  test  scores  from  the  four  most  difficult 
tests  in  one  of  the  five  Navy  schools  (N  =  117) 
served  as  an  index  of  student  performance. 
More  specifically,  the  mean  percentage  of  con 
rect  answers  on  at  least  three  of  the  four  most 
difficult  tests  was  used  as  an  index  of  student 
performance. 

The  major  reasons  for  using  this  perfor¬ 
mance  index  were:  (a)  scores  on  several  knowl¬ 
edge  tests  are  more  representative  of  academic 
performance  than  any  single  test  score;  (b) 
although  the  means  and  standard  deviations 
differed  slightly,  the  difficult  nature  of  all  the 
tests,  the  consistent  course  theme,  and  the  use 
of  percentage  scoring  allowed  a  reasonable 
combination  of  the  scores;  (c)  one  score  could 
be  missing  (e  g.,  for  a  student  who  was  dis¬ 
missed  or  set  back  to  another  class)  without 
discarding  that  poor  performer's  data;  and  (d) 
unlike  traditional  GPA  scores,  these  test  scores 
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derived  from  homogeneous  testing  and  pen 
formance  standards.  Two  other  performance 
measures  were  available.  These  dichotomous 
indices  (0  =  no;  1  =  yes)  indicated  whether 
a  student  was  assigned  to  academic  probation 
or  set  back  to  another  class,  either  event  sig¬ 
naling  poor  academic  performance. 

Results  and  Discussion 

Correlations  were  computed  between  the  16 
study  factors  and  the  three  performance  cri¬ 
teria,  the  results  of  which  are  provided  in  Table 
3  The  statistics  in  Table  3  show  that  all  but 
one  of  the  significant  correlations  were  in  the 
expected  direction.  The  exception  was  Test 
Strategy’.  Five  other  study  factors  were  related 
significantly  to  test  performance;  Competition, 
Concentration,  Memorization,  Motivation,  and 
Questioning.  Thus,  those  students  who  were 
more  competitive  or  motivated,  had  higher 
concentration  or  memorization  skills,  asked 
more  questions,  or  were  low  on  test  strategy 
skills  performed  significantly  better  on  the 
most  difficult  tests  in  this  school.  The  relatively 


large  correlations  (greater  than  .4)  for  the  Com¬ 
petition  and  Concentration  scales  should  be 
noted. 

To  explore  the  contribution  of  the  signifi¬ 
cant  study  factors  to  test  performance,  a 
regression  analysis  was  performed.  The  results 
are  shown  in  Table  3.  A  subsequent  stepwise 
regression  analysis  included  three  of  the  six 
significant  factors  and  produced  an  R( 3, 88)  = 
.618.  Thus,  the  joint  contribution  of  Compe¬ 
tition,  Concentration,  and  Test  Strategy  sig¬ 
nificantly  and  independently  accounted  for 
38.2%  of  the  test  performance  variance. 

Further,  students  who  were  placed  on  aca¬ 
demic  probation  (21.4%)  reported  low  scores 
for  two  of  the  study  factors:  Competition  (r 
=  -  .346,  p  <  .001)  and  Memorization  (r  = 
-  .206,  p  =  .031).  Also,  two  of  the  scales — 
Competition  and  Memorization — were  signif¬ 
icantly  correlated  with  both  test  performance 
and  academic  probation,  thus  providing  some 
indication  of  the  relative  importance  of  these 
study  factors  to  school  performance.  Students 
who  were  set  back  for  academic  realms 
(15.4%)  were  distinguished  only  by  their  poor 
Motivation  (r  *  -  .217,  p  =  .023). 


TABLE  3  □  Study  Factors  and  Test  Performance  Analyses 


Scale 

FOUR  TESTS 

REGRESSION  ANALYSIS 

r 

p(r) 

Bela 

t-test 

ptt) 

Anxiety 

-  000 

Competition 

443 

.000 

.398 

4  26 

.000 

Concentration 

.420 

.000 

.262 

2.81 

006 

Elaboration 

.071 

Graphic  Study  Aids 

.055 

Group  Study 

-.147 

Mastery  Beliefs 

.094 

Memonzation 

.362 

.000 

.146 

144 

153 

Motivation 

.046 

-.054 

-.511 

.611 

Organization 

.093 

Questioning 

.214 

.028 

.047 

.464 

.644 

Review 

.135 

Self-Monitonng 

.056 

Study  Resource  Management 

.075 

Test  Anticipation 

.072 

Test  Strategy 

-.213 

.031 

-.284 

-3.231 

.002 

Sole  r  «  correlation  coefficient,  jH'i  - 

probability  of  r 

under  the  null  hypothesis:  recorded  only  if  *  05  Beta  is  the 

standardized  regression  coefficient  .‘or  the  significant  variables  model  which  is  evaluated  by  the  t-test  and  its  associated 
probability 
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SECOND  VALIDITY  STUD/ 

Method 

A  second  validity  analysis  examined  the  rela¬ 
tionships  between  the  study  factors  and  test 
performance  while  controlling  for  the  effect 
of  student  ability.  This  issue  is  germane 
because  one  could  argue  that  most  or  all  of  a 
student's  test  performance  could  be  accounted 
for  bv  ability,  and  that  some  or  most  of  the 
study  factors  simply  represent  alternate  mea¬ 
sures  of  ability. 

A  good  index  of  ability  to  perform  well  in 
Navv  schools  is  provided  by  students'  compos¬ 
ite  scores  on  the  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB).  Every  enlisted  per" 
son  who  enters  the  U  S.  military  takes  the 
ASVAB,  and  the  battery  of  10  tests  has  been 
shown  repeatedly  to  serve  as  a  reliable  measure 
of  students'  academic  ability.  The  current  ver¬ 
sion  of  the  ASVAB  assesses  knowledge  and 
skills  in  10  areas:  Word  Knowledge,  Arithme¬ 
tic  Reasoning,  Coding  Speed,  Numerical 
Operations,  Math  Knowledge,  Mechanical 
Comprehension,  General  Science,  Electron¬ 
ics  Information,  Auto-Shop  Information,  and 
Paragraph  Comprehension.  Moreover,  each 
school  has  a  minimum  ASVAB  composite  cri¬ 
terion  score  for  entry  into  the  school.  Further 
details  about  the  ASVAB  can  be  found  in 
Wegner  and  Ree  (1986) 

Thus,  one  answer  to  the  question  of  whether 
ability  serves  to  moderate  the  effect  of  study 
factors  on  test  performance  is  to  describe  the 
proportion  of  variance  in  test  performance  that 
is  not  associated  with  the  ASVAB  composite 
but  is  associated  with  the  study  factors.  In 
other  words,  what  is  the  correlation  between 
each  study  factor  and  test  performance  after 
controlling  for  (or  partitioning)  the  effects  of 
student  ability? 

Subjects 

The  sample  of  301  students  who  completed 
the  Study  Factors  Survey  for  the  reliability 
analysis  was  used  for  this  analysis  Perfor¬ 
mance  data  obtained  from  the  school  con¬ 
sisted  of  test  scores  with  the  highest  failure 


rates— three  early  in  the  course,  three  approx¬ 
imately  halfway  through  the  course,  and  three 
near  the  end  of  the  course.  Not  all  subjects 
had  taken  all  of  the  tests  at  the  time  the  data 
were  received.  Scores  were  available  for  241 
students  who  completed  the  three  most  dif¬ 
ficult  tests  given  early  in  the  course,  159  who 
also  completed  the  tests  halfway  through  the 
course,  and  80  of  them  completed  the  three 
tests  near  the  course  end.  ASVAB  composite 
scores  were  available  for  most  of  the  students. 


Results 

Table  4  reports  the  results  of  the  test  perfor¬ 
mance  analysis  for  the  early  tests.  Columns 
two  and  three  of  Table  4  identify  the  signifi¬ 
cant  correlation  coefficients  between  the  study 
factors  and  the  test  performance  criterion. 
Seven  of  the  16  study  factors  were  significant. 

The  column  headed  "Partial  r"  in  Table  4 
reports  the  significant  partial  correlation  coef¬ 
ficients  between  each  study  factor  and  test  pee 
formance,  controlling  for  ability  (the  ASVAB 
composite  score).  Student  ASVAB  composite 
scores  ranged  from  191  to  268  and  had  a  mean 
value  of  233.03  and  a  standard  deviation  of 
14.65.  Four  of  the  relationships  remained 
significant  (p  <  .05;  one-tailed)  after  the 
variance  attributed  to  ability’  was  partitioned. 
These  findings  indicate  that,  irrespective  of 
their  academic  ability,  those  students  who  had 
low  anxiety  levels  as  well  as  relatively  good 
concentration  or  memorization  skills  per¬ 
formed  significantly  better  on  the  three  most 
difficult  tests  early  in  the  school.  In  addition, 
students  who  indicated  that  they  made  rela¬ 
tively  little  use  of  test  strategies  also  per¬ 
formed  better. 

To  explore  the  contribution  of  the  signifi¬ 
cant  study  factors  to  test  performance,  a 
regression  analysis  was  performed.  The  results 
are  shown  in  the  last  three  columns  of  Table 
4.  A  subsequent  stepwise  regression  analy¬ 
sis  included  three  of  the  factors  and  produced 
an  R(3,238)  =  .379.  Thus,  the  joint  contribu¬ 
tion  of  Anxiety,  Memorization,  and  Test  Strat¬ 
egy  significantly  and  independently  accounted 
for  14.4%  of  the  test  performance  variance 
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TABLE  4  □  Study  factors  and  Early  Test  Performance  Analyses 


EARLY  TESTS  PARTIAL  r  REGRESSION  ANALYSIS 


Scale 

r 

p(r) 

r 

ptr) 

Bela 

t-test 

pW 

Anxietv 

-.321 

.000 

-.230 

.001 

-.266 

-.415 

.000 

Competition 

.146 

.024 

Concentration 

.231 

.000 

.175 

.010 

.081 

1.22 

.224 

Elaboration 

.103 

Graphic  Study  Aids 

.105 

Group  Study 

-.134 

.040 

Mastery  Beliefs 

.116 

Memonzation 

.172 

.007 

.162 

.014 

.102 

.161 

.108 

Motivation 

.067 

Organization 

-.047 

Questioning 

.093 

Review 

.063 

Self-Monitoring 

.136 

.035 

Study  Resource  Management 

-.117 

Test  Anticipation 

.046 

Test  Strategy 

-.199 

.002 

-.166 

.012 

-.122 

-1.97 

.049 

Note  r  «  correlation  coefficient;  p(r)  »  probability  of  the  r  under  the  null  hypothesis  Partial  correlation  controlling  lor  the 
effect  of  ability  recorded  only  if  *  05  Beta  is  the  standardized  regression  coefficient  for  the  significant  variables  model 
which  is  evaluated  by  the  t-test  and  its  associated  probability. 


Performance  data  for  the  three  difficult  tests 
midway  through  the  course  indicated  that 
three  of  the  study  factors  were  significant. 
These  were  Anxiety  (r  =  -  .215,  p  =  .007), 
Concentration  (r  .214,  p  -  .007),  and 
Study  Resource  Management  (r  =  -  .204,  p 
=  .010)  Correlations  for  the  later  performance 
measures  indicated  that  four  study  factors 
were  significant:  Anxiety  (r  *  -.301,  p  = 
.007),  Concentration  (r  =  .261,  p  =  .020), 
Master,’  Beliefs  (r  =  .259,  p  =  .022),  and 
Questioning  (r  =  .319,  p.  =  .005).  Partial  corre¬ 
lations  were  not  performed  for  the  later  perfor¬ 
mance  measures  because  of  insufficient  data. 

DISCUSSION 

The  results  from  two  validity  analyses  indi¬ 
cate  that: 

1.  Study  factors  correlated  significantly 
with  test  performance  in  two  independent 
schools. 

2.  Different  study  factors  correlated  signifi¬ 
cantly  at  each  school. 

3  Although  more  study  factors  were  signifi¬ 
cant  at  the  beginning  of  the  course,  differ¬ 
ent  study  factors  correlated  significantly  at 
different  portions  of  the  course. 


4.  The  study  factor  relationship  to  test  per¬ 
formance  was  significant  irrespective  of  stu¬ 
dent  ability’. 

5.  Regression  analyses  supported  the  inde¬ 
pendent  contribution  of  several  study  fac¬ 
tors  to  test  performance  at  two  schools. 

When  these  validity  studies  are  viewed  to¬ 
gether,  a  pattern  emerges.  Of  the  16  study  fac¬ 
tors,  11  were  significant  in  one  or  both  schools. 
What  is  most  conspicuous  in  this  pattern  are 
the  findings  for  Memorization,  Concentration, 
and  Questioning.  Memonzation  was  related 
to  test  performance  in  both  schools,  related 
to  academic  probation,  and  survived  the  par¬ 
tial  correlation  analysts.  Likewise,  Concentra¬ 
tion  was  related  to  test  performance  in  both 
schools  and  also  survived  the  partial  correla¬ 
tion  analysis.  Because  Questioning  was  sig¬ 
nificantly  related  to  test  performance  only  late 
in  the  course,  it  did  not  undergo  the  partial 
correlation  analysis,  but  was  significant  in 
both  schools.  These  study  factors  deserve 
attention,  at  least  for  technical  training  schools. 
Moreover,  these  study  factors  may  have  impli¬ 
cations  for  the  education-technology  interface 
in  that  technology  may  be  able  to  affect  how 
students  memorize,  concentrate,  and  question 
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The  findings  for  Anxiety  and  Motivation 
may  have  been  school  specific.  Anxiety  was 
related  to  all  three  performance  measurements 
in  one  school,  and  it  also  survived  the  partial 
correlation  analysis.  In  the  other  school,  Moti¬ 
vation  was  related  to  both  test  performance 
and  academic  probation.  It  is  not  dear  whether 
Anxiety  and  Motivation  are  antithetical  or  sim¬ 
ply  school  sperific.  Other  study  factors  either 
did  not  pass  the  partial  correlation  test  (Com¬ 
petition,  Group  Study,  Self-Monitoring)  or 
were  not  subjected  to  it  (Mastery  Beliefs  and 
Study  Resource  Management).  Also,  although 
Competition  failed  the  partial  correlation  test, 
its  affects  should  be  investigated  further  due 
to  the  influence  of  this  factor  in  military  train¬ 
ing.  This  topic  deserves  further  study. 

Reasoning  suggests  that  factors  such  as 
Anxiety,  Concentration,  Memorization,  and 
Questioning  have  important  ties  to  the  study- 
learning-performance  process.  These  analyses 
provide  solid  evidence  fur  the  consideration 
of  such  factors  in  learning  environments.  Only 
three  study  factors  provided  a  significant 
result  in  an  unexpected  direction— Group 
Study  (which  did  not  pass  the  partial  corre¬ 
lation  analysis).  Study  Resource  Management, 
and  Test  Strategy.  The  item  content  for  the  Test 
Strategy  scale  concerns  the  order  of  answer¬ 
ing  easy  versus  difficult  questions  on  a  test. 

It  may  be  that  in  these  and  similar  schools  it 
is  necessary  or  functional  to  answer  test  ques¬ 
tions  from  first  to  last  rather  than  answer  the 
easy  questions  first.  This  topic  also  deserves 
further  study. 

The  three  key  issues  associated  with  study 
skills  research  are  their  selection  and  develop¬ 
ment,  training,  and  effectiveness  assessment 
(Dansereau,  1985).  This  article  addressed  two 
of  those  issues  in  a  military  context.  Based  on 
these  findings,  evaluations  should  be  unden 
taken  to  determine  whether  study  factor  train¬ 
ing  can  be  implemented  to  influence  test 
performance.  For  example,  training  may  be 
indicated  for  students  whose  scores  fall  below 
selected  cutoff  points  for  Concentration,  Mem¬ 
orization,  Questioning,  or  other  skills.  Dob- 
rovolny,  McCombs,  St  Judd  (1980),  Hoffman 
et  al.  (1988),  and  McCombs  St  Dobrovoiny 
(1982)  are  resources  that  describe  military 
training  aids  for  several  study  factors,  includ¬ 


ing  Reading  Comprehension,  Concentration, 
Memorization,  and  Motivation.  A  fourth  key 
issue  that  should  be  raised  and  addressed  by 
future  research  is  whether  study  skills/factors 
are  compensatory  or  necessary.  Two  regres¬ 
sion  analyses  provided  evidence  that  some  of 
these  study  factors  made  independent  con¬ 
tributions  to  test  performance.  Yet,  if  students 
are  low  on  one  or  a  few  factors,  can  other  fac¬ 
tors  compensate  for  that  deficit?  Such  issues 
require  dedicated  research  effort  if  we  are  to 
make  advances  into  understanding  the  study- 
leam-performance  process  and  how  technol¬ 
ogy  can  contribute  to  that  process. 

Although  the  NPRDC  Study  Factors  Survey 
was  designed  to  assess  study  factors  in  mili¬ 
tary  technical  schools,  the  factors  obtained  in 
this  research  have  high  face  validity,  seem  to 
be  generic,  and  thus  may  underlie  learning 
and  study  in  the  general  population.  Conse¬ 
quently,  the  findings  from  this  research  effort 
may  have  applications  to  non-military  aca¬ 
demic  settings.  Moreover,  research  is  required 
to  delineate  whether  different  learning  objec¬ 
tives  or  different  topics  of  varying  content  and 
difficulty  require  different  study  factors  for 
effective  learning.  □ 

REFERENCES 

Bachelor,  P.,  k  Michael,  W  B  (1988).  The  compa¬ 
rability  of  the  factonal  validity  of  the  Study  Atti¬ 
tudes  and  Methods  Survey  in  a  high  school 
sample  and  in  a  university  sample  Educational  and 
Psychological  Mtasurmtnt.  48. 987-994. 

Bartlett,  A.,  k  Knoblock,  L.  (1988)  A  self  report 
questionnaire  for  updating  college  students'  study 
skills,  lournal  of  Reading.  31, 361-364. 

Brown,  A.  L.,  Brantford, ).  D  ,  Ferrara,  R.  A.,  k 
Cam  pi  one,  J.  C.  (1983).  Learning,  remembering, 
and  understanding  In  P  H.  Mussen  (Ed  ),  Hand¬ 
book  of  child  psychology:  Vol.  Ill  Cognitive  develop¬ 
ment  (4th  ed,  pp  77-166).  New  York:  Wiley. 
Dansereau,  D.  F.  (1985)  Learning  strategy  research. 

In  J.  W.  Segal,  S.  F  Chipman,  k  R.  Glaser  (Eds. ), 
Thinking  and  learning  skills  (Vol  1,  pp.  209-239). 
Hillsdale,  NJ:  Lawrence  Erlbaum. 

Dansereau,  D.  F.,  Actkmson,  T  R  ,  Long,  G  L  ,  k 
McDonald,  B,  A.  (1974).  Learning  strategies.  A 
review  and  synthesis  of  the  current  literature  (Tech¬ 
nical  Report  74-70,  AD-A007-722)  Brooks  Air 
Force  Base,  TX;  Air  Force  Systems  Command. 
Dansereau,  D.  F.,  McDonald.  B  A  ,  Collins,  K.  W., 
Garland, ). C.,  Holley,  C.  D.,  Diekhoff,  C  M.,4 


9 


28 


ETOiD.  VtM  39  NO  2 


Evans.  S  H  (1979).  Evaluation  of  a  learning  strat¬ 
egy  system  In  H.  F.  O'Neil  4c  C.  D.  Spielberger 
l  Eds  ),  Cogmtne  and  affect  ne  learning  strategies  (pp . 
3—43)  New  York:  Academic  Press. 

Davou.  D.,  4c  McKelvie,  S.  J.  (1984).  Relationship 
between  study  habits  and  performance  on  an 
intelligence  test  with  limited  and  unlimited  time. 
Psychological  Reports,  54,  367-371. 

Deny,  S  .  J  .,4c  Murphy,  D.  A.  (1986).  Designing  sys¬ 
tems  that  train  teaming  ability:  From  theory  to 
practice  Review  of  Educational  Research,  56, 1-39. 

Dobrovolny.  J  L.,  McCombs,  B.  L.,  &  Judd.  W.  A. 
(1980).  Study  skills  package:  Development  and  evalu¬ 
ation  (AFHRL-TR-79-43)  Brooks  Air  Force  Base, 
TX.  Air  Force  Human  Relations  Laboratory 

Gadzella  B  M  (1982)  Computer-assisted  instruc¬ 
tion  on  study  skills  Journal  of  Experimental  Edu¬ 
cation,  50,  122-126 

Hoffman,  L.  M  .  Hahn,  C.  P,  Hoffman,  D.  M.,  4c 
Dean,  R  A.  (1988)  Evaluation  of  the  fob  Skills  Edu¬ 
cation  Program  Learning  outcomes  (AR1  Research 
Note  88-99)  Alexandria.  VA:  U  S.  Army  Institute 
for  the  Behavioral  and  Social  Sciences. 

McCombs,  B  L..  k  Dobrovolny,  J  L.  (1982)  Stu¬ 
dent  motivational  skill  training  package:  Evaluation 
lor  Air  Force  technical  training  (AFHRL-TP-82-31). 
Lowrv  Air  Force  Base,  CO  Air  Force  Human  Rela¬ 
tions  Laboratory 

Nadson,  J  S  ,  Michael  W.  B  ,4c  Michael,  J.  J  (1988). 
The  factonal  and  concurrent  validity  of  a  revised 
form  of  the  Study  Attitudes  and  Methods  Sur¬ 
vey  (SAMS)  Educational  and  Psychological  Measure¬ 
ment.  48,  969-976 

O'Neil,  M  J.,4c  Child.  D.  (1984)  Biggs'  SPQ  A  Bnt- 
ish  study  of  its  internal  structure.  British  Journal 
of  Educational  Psychology.  54.  228-234. 

Pintnch,  P  R  .  McKeachie,  W,  J  ,  Smith,  D  A., 
Dolianac.  R  .  Lin.  Y  ,  Naveh-Ben)amin,  M., 
Crooks  T  ,  k  Karabenick,  S.  (1988)  Motivated 
Strategies  for  Learning  Questionnaire  IMSLQ)  Un¬ 
published  manusenpt.  University  of  Michigan, 
National  Center  for  Research  to  Improve  Post¬ 
secondary  Teaching  and  Learning. 

Robinson,  F  P  (1946).  Effective  study.  New  York: 
Harper  and  Brothers. 

Schulte.  A  C  ,  4c  Weinstein,  C.  E.  (1981,  April). 
Inventories  to  assess  cogmtne  learning  strategies  Paper 
presented  at  the  meeting  of  the  Amencan  Educa¬ 
tional  Research  Association,  Los  Angeles,  CA. 


Spielberger,  CD,  Gonzalez,  H  P,  k  Fletcher,  T. 
(1979).  Test  anxiety  reduction,  learning  strategies, 
and  academic  performance  In  H.  F  O'Neil  k  C 
D.  Spielberger  (Eds  ),  Cogmtne  and  affecttie  learn ■ 
mg  strategies  (pp.  111-131).  New  York  Academic 
Press. 

Sbcht,  T.  G.  (1979).  Developing  literacy  and  learn¬ 
ing  strategies  in  organizational  settings.  In  H  F. 
O’Neil  k  C.  D.  Spielberger  (Eds ),  Cogmtiveand 
affectne  learning  strategies  (pp.  275-307).  New  York 
Academic  Press. 

Wade,  S.E.,  4c  Trathen,  W  (1989)  Effect  of  self- 
selected  study  methods  on  learning.  Journal  of  Edu¬ 
cational  Psychology,  81, 40-47 
Wegner,  T.  G.,  k  Ree,  M. ).  (1986)  Alternative  armed 
forces  qualification  test  composites  (Technical  Report 
86-27)  San  Antonio,  TX.  Air  Force  Human  Rela- 
bons  Laboratory. 

Weinstein,  C.  E.,  k  Mayer.  R.  E.  (1986)  The  teach¬ 
ing  of  learning  strategies.  In  M  C  Wittrock  (Ed), 
Handbook  of  research  on  teaching  (3rd  ed..  pp 
315-327).  New  York:  Macmillan 
Weinstein,  C.  E.,  4r  Underwood,  V.L  (1985)  Learn¬ 
ing  strategies  The  hew  of  learning  In  J.  W  Segal. 
S.  F.  Chipman,  4c  R.  Glaser  (Eds  ),  Thinking  and 
learning  skills  Vol.l.  Relating  instruction  to  research 
(pp  241-258).  Hillsdale,  NJ:  Lawrence  Erlbaum 
Weinstein,  C  E.,  Underwood,  V  L  .  Rood,  M  M  , 
Conolon,  C.  M..  Wild.  M.,  4c  Kennedy.  T  (1980) 
The  effects  of  selected  instructional  variables  on  the  ac¬ 
quisition  of  cognitive  leami  ig  strategies  (Technical  Re¬ 
port  463).  Alexandna,  VA:  U  S.  Army  Research 
Insritute  for  the  Behavioral  and  Social  Sciences 
Weinstein,  C.  E.,  Washington,  T.  P,  Wicker,  F.  W , 
Duty,  D  C  .  4c  Underwood.  V  L  (1980)  Theefkcts 
of  material  and  task  vacations  on  brief  cogmtne  learn¬ 
ing  strategies  training  program  (Technical  Report 
461)  Alexandna.  VA:  U.S.  Army  Research  Insti¬ 
tute  for  the  Behavioral  and  Social  Sciences 
Wilson,  L.  S  (1990),  An  on-line  prescnption  for 
basic  skills  Training  b  Development  Journal  44 
36-41 

Wittrock,  M.  C.  (1974).  Learning  as  a  generative  pro¬ 
cess.  Educational  Psychologist,  11, 87-95. 

Wittrock,  M.  C.  (1979).  Applicabons  of  cognibve 
psychology  to  educabon  and  training.  In  H  F 
O'Neil  4c  C.  D.  Spielberger  (Eds  ),  Cogmtne  and 
affective  learning  strategies  (pp  309-327)  New  York 
Academic  Press. 


10 


DISTRIBUTION  LIST 


Distribution: 

Chief  of  Naval  Operations  (MP&T)  (OP-01) 

Chief  of  Naval  Education  and  Training  (Code  N-32) 

*  Defense  Technical  Information  Center  (DT1C)  (2) 

Copy  to: 

*  Office  of  Chief  of  Naval  Research  (OCNR- 10).  (ONT-222),  (ONT-20) 

Chief  of  Naval  Education  and  Training  (Code  N6),  (Code  00),  (Code  01),  (Code  L01)  (6) 
Curriculum  and  Instructional  Standards  Office,  Fleet  Training  Center,  Norfolk,  VA 
Head,  Training  Assessment  Division  (PERS-1 16) 

Commander,  Naval  Training  Center,  San  Diego,  CA 

Commanding  Officer,  Service  School  Command,  San  Diego,  CA  (Code  3200) 

Commanding  Officer,  Service  School  Command,  San  Diego,  CA 

Department  of  the  Air  Force,  DET  S,  Armstrong  Laboratory  Directorate,  Brooks  Air  Force  Base,  TX 
TSRL/Technical  Library  (FL  2870) 

Director  of  Research,  U.S.  Naval  Academy 


i 


11 


